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Abstract 

Teacher education programs in the State of California are increasingly held accountable 
for the assessment of teacher candidates’ competencies and performance. One salient assessment 
tool is the utilization of portfolios in teacher education programs (Wolf, 1989; 1991). Among the 
newest and most innovative templates of portfolios in teacher education are electronic portfolios. 
Electronic portfolios are gaining popularity in part, due to the prevalence of their technological 
accessibility and the numerous on-line portfolio options. 

This paper reports on a pilot project that utilized electronic portfolios as an assessment 
tool at a start-up public university across two fifth-year credential programs respectively: the 
Education Specialist: Mild/Moderate Disabilities Level One and the Single Subject Credential 
Programs. The primary research question addressed was how do the constituents (e.g., credential 
candidates, faculty, and portfolio evaluators) perceive the electronic portfolio processes within 
and across the two teacher preparation programs. 

Descriptive findings related to the development and management of electronic portfolios 
within and across programs is shared. A discussion of the comparative findings from the 
programs is presented, as well as the implications for the implementation of electronic portfolios 
in teacher education programs. 




A Tale of Two Programs 3 



A Tale of Two Programs: 

A Comparative Study of Electronic Portfolio Assessment in Teacher Education 

Introduction 

Teacher education programs in the State of California are increasingly being held 
accountable for the assessment of teacher candidates’ competencies and performance. One 
component of the teacher education assessment movement in California is the implementation of 
the Teacher Performance Assessment (TP A) in selected credentialing areas including, the 
multiple and single subjects. In the interim, before the statewide TPA is mandated uniformly for 
all of California’s teacher credentials, teacher education programs have autonomy in decisions 
about their own teacher candidate assessment processes and outcome measures. 

Nationally and internationally, portfolios are highly regarded as a salient assessment tool 
in teacher education (Wolf, 1989; 1991). Research on the use of portfolios in pre-service teacher 
preparation programs reveals that portfolios are a medium through which student teachers reflect 
on their professional development (Loughran & Corrigan, 1995). 

In a study of the preparation of teachers of students with behavioral disorders, Bloom and 
Bacon (1995) found that portfolios assisted the students with self reflection and assessment of 
their development as a professional educator. The authors found that the portfolios provided 
assistance to the student teachers in the development of a wide range of skills to aid in decision 
making, problem solving, and establishing a connection with the teaching profession. The 
portfolio process also gave students more self confidence in their abilities through the 
responsibility and control over their own work. The students were energetic and enthusiastic 
about having the opportunity for choice in the development of their portfolios, which led to 
addressing complex issues in teaching instead of writing for faculty approval. Some 
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disadvantages included students’ apprehension with the evaluation process, as well as the labor 
intensiveness of the assessment process for the faculty. The authors suggested that these issues 
may be managed by the active involvement of participating students and faculty in the portfolio 
process, as well as the articulation of clear expectations about the assessment process and 
required products. 

In a study of the use of portfolios as a tool for promoting teaching, Klenowski (2000) 
focused on the extent to which portfolios supported the development of reflective practice and 
teaching among students and lecturers in secondary education. It was found that the use of 
portfolios profoundly impacted the pedagogic practice among the pre-service teachers, but also 
among the lecturers. It was reported that initially the students did not understand the value of the 
portfolio; they viewed it as a data Thing system. Over time, the assessment procedures and 
explanations became clearer; the students began to appreciate the reflective aspects of the 
portfolio process. Portfolio exemplars were provided, as well as more specific evaluation criteria. 
Lecturers also gained from embedding the portfolio process into their courses. Changes in their 
teaching styles were noted through the integration of reflective practice into their teaching and 
the connection to their own philosophy of teaching. 

In the field of teacher education, electronic portfolios are gaining in use due to the 
prevalence of technological accessibility and numerous on-line portfolio options. Dysthe and 
Engelsen (2004) argued that digital portfolios can offer a number of benefits, including a great 
degree of peer interaction among student teachers. If a student is given access to review and 
critique other students’ portfolios, the social context of learning may be exemplified as the 
ability to share and revise work is easily facilitated in digital portfolio systems. 




A Tale of Two Programs 5 



Among the advantages and disadvantages of electronic portfolios are the technologies 
and technological skills required for such systems. Woodward and Nanlohy (2004) examined 
how student teachers and faculty developed and utilized a digital portfolio rather than a 
traditional paper format. Initial results indicated that the students encountered problems learning 
the new software, navigating copyright restrictions on music and photographs, and managing the 
storage of large amounts of data. The advantages found in using the digital portfolios were more 
interaction with the intended audience and the easy reproduction of the electronic portfolio. The 
authors recommended that digital portfolios should be developed within a carefully designed 
framework. 

The aforementioned studies present many relevant issues related to the use and 
management of traditional and electronic versions of portfolios in teacher education programs. 
There are many benefits gained from the use of portfolios, as well as issues for future research to 
investigate. To this regard, this paper reports on a pilot project that utilized electronic portfolios 
as an assessment tool in two fifth-year credential programs: the Education Specialist: 
Mild/Moderate Disabilities Level I and the Single Subject Credential Programs. 

The primary research question guiding this study was: How do the constituents (e.g., 
credential candidates, faculty, and portfolio evaluators) perceive the electronic portfolio 
processes within and across the two teacher preparation programs? Descriptive findings related 
to the development and management of using electronic portfolios in each program will be 
shared. A discussion of the comparative findings from the programs is presented and the 
implications for electronic portfolio implementation in teacher education programs are discussed. 
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Method 

Background 

Our start-up public university opened its doors to undergraduate and post-baccalaureate 
students in August of 2002. At that time, the elementary Multiple Subjects Credential Program 
enrolled 20 students. During the following academic year, the secondary Single Subject 
Credential and the Education Specialist: Mild/Moderate Disabilities Level I Credential Programs 
began with 8 and 12 students, respectively. All three programs were written and approved under 
the newly developed state of California teacher education policy standards kn own as SB 2042. 
Included in the new state policy standards are 13 Teacher Performance Expectations (TPEs) that 
every credential student is expected to demonstrate proficiency when exiting their respective 
credential program. 

Likewise, teacher candidate assessment was a mandated standard for all programs in the 
State of California. The California Commission on Teacher Credentialing (CCTC) designed the 
Teacher Performance Assessment (TP A) as a measure to assess all multiple subjects and single 
subject candidates on their mastery of the 13 TPEs. 1 However, funding was not provided for the 
four-task TPA implementation. Therefore, every teacher preparation program had to design and 
implement its’ own equivalent assessment plan. At our university, we decided to conduct a pilot 
project using portfolio assessment as a tool across our credential programs to determine the 
efficacy of our system while taking into account the current state and national accreditation 
standards, as well as the proposed public policy changes. 

The assessment process began in fall of 2003 as a pilot project for the Multiple Subjects 
Credential Program. Five full-time tenure-track and two full-time lecturers in the Education 

1 At the time of this pilot project, the Education Specialist: Mild/Moderate Disabilities Credential was not required 
to be aligned to or assess the TPEs. 
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Program participated in four meetings in the fall of 2003 and five meetings in the spring of 2004. 
These meetings resulted in the identification of four core competencies that our teacher 
candidates should acquire and demonstrate upon program completion. Each of the 13 TPEs were 
backward-aligned with one of the four core competencies. 

Portfolio development and requirements 

The Education Specialist: Mild/Moderate Disabilities Level 1 Credential and the Single 
Subject Programs designed their own core competencies with supporting elements from program 
hallmarks, state and national professional development standards, and a review of the literature. 
See Appendices A and B for each program’s portfolio structure, core competencies, and 
supporting sub-elements. One tenure-track faculty member in each program served as the 
Faculty Assessment Coordinator (FAC) and took the lead on implementing the electronic 
portfolio assessment project for their respective programs. The FACs consulted with each other 
on technical aspects of the electronic portfolio development, and also worked collaboratively to 
create common rubrics for artifacts and reflective summaries of the portfolios; these efforts 
allowed for the aggregation and comparison of data across the credential programs. 

In each program, credential candidates were introduced to the electronic portfolio 
commercial program selected by the FACs, the portfolio procedures, and portfolio requirements 
in the respective student teaching seminars. Education Specialist candidates were required to 
submit a reflective summary for each core competency and a minimum of one artifact for each 
core competency sub-element. Single Subject candidates were required to include a minimum of 
one artifact for each core competency sub-element and one summary reflection for each core 
competency for evaluation. At least 50% (n=l 1) of the artifacts in the Single Subject candidates’ 
portfolio were required to be original student work. 
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Research site and participants 

This study took place at a four-year public university in Southern California. Twenty 
credential candidates and six evaluators participated in the Education Specialist pilot assessment 
project. Twelve credential candidates and five evaluators participated in the Single Subject pilot 
assessment project. All credential candidates signed informed consent forms and agreed to 
participate in this study. 

Data collection and analysis 

Because this study was descriptive in nature and detailed the processes of piloting an 
electronic portfolio assessment project, qualitative data were collected. Detailed field notes were 
taken at all interviews, meetings, and courses when the portfolios were a topic of discussion. 
Focus group interviews with evaluators in each program were conducted after the evaluation of 
candidate portfolios was complete. Data from candidates were obtained during student teaching 
seminars both during the portfolio process and after their portfolios were electronically 
submitted. E-mail correspondence with students and evaluators was used as an additional data 
source. The two FACs kept notes and journal logs pertaining to the electronic portfolio 
assessment project. 

Each program had evaluation procedures in common and those that were unique to the 
program. In the Education Specialist Program, pairs of evaluators were assigned to a common 
group of candidates and they independently evaluated all required elements of the portfolios. The 
evaluators used rubrics developed for the assessment project (Grier & Denney, 2004). See 
Appendix C for the reflective summary rubric. Inter-rater reliability was established among pairs 



of evaluators. 
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In the Single Subject Program, a meeting with all evaluators occurred to establish base- 
line understandings of the reflective narrative rubric and how this applied to evaluating student 
work (inter-rater reliability) with the electronic portfolio system. The evaluators identified areas 
of expertise within the portfolio sub-elements and evaluated all student portfolios in their 
expertise area. Evaluators scored portfolio artifacts independently at their own convenience over 
winter break. They reviewed candidate submissions electronically, but used paper and pencil 
scoring sheets because the on-line system could not support multiple evaluators scoring 
simultaneously. 

Each FAC independently analyzed their own program data. After the initial coding was 
conducted, the two faculty members discussed and compared their findings. The comparative 
findings were further analyzed for emerging, over-arching themes across the two programs. 

Results 

Two major themes emerged from the data analysis. These themes were identified as 
process and tool oriented elements across evaluator, student, and FAC constituencies. In Tables 
1 and 2, the categories of data are presented that summarize the results of the study. Table 1 
presents process-oriented elements and Table 2 presents tool-oriented elements within and across 
the two credential programs. 
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Table 1 . Process-Oriented Elements Within and Across the Credential Programs 



Process- 

Oriented 

Elements 


Evaluator-generated 


Student-generated 


FAC-generated 


Education 

Specialist 


• Liked the evaluation rubrics, 

however, would suggest 
changing the artifact rubric 
to a binary schema e.g., 
meet criteria, does not meet 
criteria 

• Concern about the artifacts 

and the validity of 
candidates’ original work 
vs. secondary sources 

• Need to create a more 

comprehensive student 
assessment mechanism 






Common 


• Too much time for 

evaluation 

• 70+ hours for 20 students 

for one semester in 
Education Specialist; 60+ 
hours for 1 2 students for 
one semester in Single 
Subject 


• Timing of portfolio 

submission (end of 
semester) seemed very 
rushed 

• Wanted more information 

on criteria of elements and 
what kinds of artifacts 
“count” 

• Uncertainty and anxiousness 

over “who” was going to 
be evaluating portfolios 

• Too much time was taken 

out of student teaching 
seminar to talk about 
portfolios 

• Portfolio seems like an “add 

on” requirement 


• Too much time to coordinate 
all components between 
students and evaluators 


Single 

Subject 


• Liked focusing on one 

element at a time across 
students 

• Needed to have a context for 

student artifacts submitted 
(where is the connection?) 

• Wanted more group training 

on the portfolio criteria & 
purpose 

• Most artifacts were class 

assignments— why re- 
evaluate? 
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Table 2 . Tool-Oriented Elements Within and Across the Credential Programs 



Tool- 

Oriented 

Elements 


Evaluator-generated 


Student-generated 


FAC-generated 


Education 

Specialist 








Common 


• The commercial program was 

very hard to use e.g., home 
computer speed and 
compatibility 

• Opening the artifacts was 

difficult especially with 
scanned documents 

• Too much time to evaluate e- 

portfolios 

• Was convenient to evaluate at 

home and “come and go” 
from the task 

• Time on technical use was 

excessive 


• Liked being able to submit 

artifacts electronically 

• System was very easy to use 

• Liked being able to link 

lesson plans created in the 
commercial program 
directly into portfolio 

• Access to a scanner was 

sometimes problematic 


• Multiple simultaneous 

evaluators (only one 
evaluator can have access 
to artifact at a time) 

• Once students send artifacts 

in for “evaluation” they 
are locked out of portfolio 

• Can aggregate student scores 

but not reviewer 
comments 


Single 

Subject 






• Part-time student work was 

not evaluated— sent for 
review 

• Reflective narratives were 

not evaluated because of 
the system set-up 

• Primary source requirement 

not evaluated wholistically 



Discussion 

Both teacher education programs made significant changes to their respective portfolio 
assessment processes based upon the results of this study. The Education Specialist Program 
faculty made three significant changes to the assessment of credential candidates using the 
electronic portfolio system. First, the content and development for each core competency of the 
portfolio will be embedded into coursework and fieldwork, and not as an additional requirement 
for program completion. Secondly, signature assignments will be identified and explicitly 
outlined in the program’s coursework and fieldwork for the students. These signature 
assignments are aligned to the portfolio elements. The content of the student portfolios will be 
expanded to include signature assignments, reflective summaries for the core competencies, and 
student teaching evaluations -all of which must be original work. Lastly, the portfolio process 
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was previously developed during the culminating semester when they would graduate -creating a 
tremendous amount of time and work for the evaluators and assessment coordinator at one point 
of time in the program. Since the portfolio process has now been embedded into all of the 
coursework and fieldwork throughout the program, students will begin the portfolio process at 
the inception of their program; therefore creating a greater sense of shared accountability for the 
portfolio assessment process across the program’s faculty and students. The signature 
assignments to be included in the portfolios will receive a formative evaluation in the designated 
courses and a summative evaluation in the students’ culminating semester of the program. These 
aforementioned changes specifically addressed comments from the FAC, students, and 
evaluators regarding the identified obstacles of the portfolio process reported in this study. 

The Single Subject Program faculty recommended three major changes in the portfolio 
process as a result of this study. First, artifact submission will be embedded into the program 
coursework and evaluated by the individual course instructors as part of the course requirements. 
The Single Subject Program instructors have since identified course assignments that align to the 
portfolio elements that will be considered signature assignments for use as possible portfolio 
artifacts. This list of suggested artifacts (e.g., signature assignments) will be given to students 
when entering the program and will be identified in each course syllabus. Second, all artifacts 
submitted by students must be their original work. Now that many of the process details and 
signature assignments have been identified and integrated into the coursework, the Single 
Subject faculty believes it reasonable for students to generate artifacts that represent their 
knowledge, skills, and dispositions. Third, the structure of the portfolio evaluation process has 
been changed. Only the reflective narrative will be evaluated at the end of each semester. This 
reduces not only the time spent by evaluators but also the cost necessary for paying evaluators. 
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First semester student teachers will submit all of the artifacts and a reflective narrative for one 
core competency area for formative evaluation. Second semester student teachers (e.g., 
graduating students) will submit a complete portfolio for summative evaluation. These changes 
addressed the concerns of the FAC, students, and evaluators regarding the portfolio process. 

Interestingly, each program is adopting the practice of signature assignments to embed 
the portfolio process into the programs’ structure but will be doing so in different ways. The 
signature assignments in the Education Specialist portfolio will be the same for each student. 
Therefore, every student portfolio will have the same artifacts included in their portfolio. This is 
not the case for the Single Subject portfolios. The signature assignments are identified as only 
suggestions for portfolio elements; there is freedom for the students to include other original 
artifacts in lieu of a signature assignment. Because the single subject students do not all take the 
same sequence of classes due to the specialized content methods classes and because the faculty 
feel it is more important to see how the students construct their own understandings of the core 
competencies and make connections in their reflective narrative, this choice was built into 
artifact selection. This is an important distinction between the two programs. 

There were significant concerns raised by evaluators and the FACs about the commercial 
electronic portfolio program. Although the evaluator and FAC comments from each program 
were similar, the Education Specialist program decided not to use the commercial program the 
following year based upon the difficulties and obstacles identified in this study. The Single 
Subject program however, continued to use the commercial product based upon the positive 
feedback received from the students regarding ease of use and the access to the California State 
Academic Content Standards within the lesson plan template. The FACs shared the data and 
initial results from this study with the e-portfolio company. The company has since updated their 
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service to include options for multiple simultaneous raters and aggregating qualitative comments 
on-line. We feel this is a significant outcome of this study. The Education Specialist program is 
now reconsidering using the system now that these two major obstacles have been remedied. 

It is anticipated that these changes in the portfolio processes in both programs will 
become seamless and not viewed as an additional assignment giving validity to the portfolio as a 
significant program requirement. These changes will also make program instructors more aware 
and accountable for evaluating the portfolio products as part of the curriculum and not a time- 
consuming event at the end of the semester. As with any form of assessment, time will be needed 
in learning to use the tools. However, as evaluators use the tools more frequently, less time will 
be involved in assessing student work. 

Implications 

Overall, a portfolio must be and perceived as integrated into a teacher education program 
by students and faculty. It is important that portfolio elements are aligned and goals and tasks are 
clear to the participants. Additionally, the time and resources needed to successfully implement 
and maintain a portfolio system must not be overlooked. Coordinating the players and tasks, 
evaluation of candidate work, evaluation of process, aggregation and analysis of data, data 
management, and maintenance of the process are all necessary elements in a successful portfolio 
assessment process. 

The significance of this pilot study, as compared to other institutions implementing 
electronic portfolios for the first time, is that being at a start-up institution is an extremely unique 
environment. As faculty, we had the freedom to create curricular programs and assessment tools 
without preconceived structures and historical issues to overcome. This freedom exposes other 
obstacles - ones that may be representative of any teacher education program as evidenced in the 
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literature. At the time of this study, we were not just trying to develop and implement an 
electronic portfolio assessment system in an established institution; we were (and still are) 
simultaneously developing curricula and policies for the credential programs and for the 
university. Ours is a fluid and flexible environment and one in which our students are learning 
the same traits as they begin the learning to teach process. At first, the students in both programs 
viewed the portfolio as an “add-on” piece — because for them, it was not integrated into the 
program. But, since this study, the portfolio systems in both programs have evolved in different 
ways and all constituents have a better understanding of the portfolio and its significance in their 
professional lives (Klenowski, 2000). 
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Appendix A 

Education Specialist: Mild/Moderate Disabilities Level I Credential Program Portfolio 
Organizational Outline: 

I. Resume 

II. Philosophy of Teaching 

III. Core Competency: Foundations 

A. History 

B. Laws and Policy 

C. Ethics 

D. Professional Standards and Practices 

E. Family Systems Across the Life Span 

F. Service Delivery Systems 

G. Consultation Models and Processes 

H. Effective Communication and Collaboration 

I. Characteristics of Learners 

IV. Core Competency: Assessment 

A. Individual Assessment 

B. Group Assessment 

C. Assessment Processes - Pre-referral, Referral, Identification, 

Evaluation, Re-evaluation 

D. IEP Goals and Objectives 

E. Behavioral Assessment 

V. Core Competency: Methods 

A. Learning Environments, Social Interaction, and Classroom 
Management 

B. Core Curriculum in General Education 

C. Specialized Curriculum 

D. Instructional Methods 

E. Intervention Methods 

F. Modification of Methods and Materials 

G. Positive Behavioral Support 

VI. Guiding Questions: 

1 . What are the special qualities that you bring as an individual and you have developed 
as a professional? 

2. How does this portfolio represent you as a developing professional who can address 
the diverse needs of all students? 

3. How does this portfolio represent your evolving philosophy of teaching and practice? 
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Appendix B 

Single Subject Teaching Credential Program Portfolio Organizational Outline: 

I. Resume 

II. Philosophy of Teaching 

III. Core Competency: Learning Environment (TPEs 9, 10, 11) 

A. Establishing and Maintaining a Culture for Learning 

B. Classroom Procedures 

C. Student Behavior 

D. Long-term Planning and Instructional Goals 

E. Daily Planning and Lesson Objectives 

F. Resource Selection, Adaptation, and/or Enrichment 

IV. Core Competency: Instructional Process (TPEs 1, 2, 3, 4, 5, 7) 

A. Content Knowledge in the Content Area — Grade/Developmentally Appropriate 

B. Instructional Methods in the Content Area — Grade/Developmentally Appropriate 

C. Literacy in the Content Area 

D. Teaching English Learners 

E. Teaching Special Needs Learners 

F. Student Engagement in Learning 

G. Assessment of Student Learning 

V. Core Competency: Learning About Students (TPEs 6, 8) 

A. Approaches to Learning 

B. Background and Culture 

C. Behavioral Assessments 

D. Assessing Literacy Skills 

E. Assessing Special Needs 

VI. Core Competency: Professionalism (TPEs 12, 13) 

A. Professional, Legal, Ethical Obligations 

B. Reflecting on Teaching 

C. Communication with Families 

D. Maintaining Accurate Records 

VII. Portfolio Summary (graduating students only) 

1 . What are the special qualities that you bring as an individual and you have developed as a 
professional? 

2. How does this portfolio represent you as a developing professional who can address the 
diverse needs of all students? 

3. How does this portfolio represent your evolving philosophy of teaching and practice? 
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Appendix C 



Reflective Narrative Rubric 





Distinguished 


Proficient 


Emergent 


Unsatisfactory 


Value: 3 


Value: 2 


Value: 1 


Value: 0 


Knowledge, 
Skills, & 
Dispositions 


Reflected in-depth 
on knowledge skills 
and dispositions 
related to core 
competency. 


Reflection on 
knowledge, skills, 
and dispositions 
related to core 
competency. 


Reflected on at least 
two (knowledge, 
skills, or 

dispositions) related 
to core competency. 


Little to no 
reflection present 
when addressing 
knowledge, skills 
and dispositions 
related to core 
competency. 


Connected to 
TPEs 


Each TPE for the 
core competency is 
addressed with 
several examples 
from portfolio 
artifacts. 


Each TPE for the 
core competency is 
addressed with at 
least one example 
from portfolio 
artifacts. 


Most TPEs for the 
core competency are 
addressed with at 
least one example 
from portfolio 
artifacts for each 
TPE. 


Most TPEs are not 
addressed or no 
examples from 
portfolio artifacts 
are referenced. 


Writing: 
Grammar & 
Mechanics 


Errors in grammar 
and mechanics are 
not present. 
Narrative is 
extremely well- 
written. 


Few errors in 
grammar and 
mechanics are 
present. Narrative is 
well-written. 


More than a few 
errors in grammar 
and mechanics. 
Writing needs 
improvement. 


Narrative needs 
extensive revisions 
due to grammar and 
mechanics errors. 



